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ABSTRACT 

The basic rationale for this study was that attempts 
to measure the service capabilities of public libraries, either in 
relation to the needs of their communities or to the standards 
adopted by the professionr are hampered by the lack of criteria of 
quality or effectiveness. It was recognized that new measures were 
needed to gauge effectiveness while eliminating extraneous factors, 
specif icallyr these two phases reported upon here were designed to 
(1) identify measurement criteria which would discriminate among 
public library services, (2) develop an operational methodology 
whereby local librarians could collect the necessary data for 
processing and refinement, and (3) demonstrate that the measurement 
criteria could be put into a theoretical model which would provide a 
professional basis for assessing the activity of basic programs. The 
following criteria were studied: description of collectiour building 
usage, circulation, facilities usage, patterns of reference usage ^ 
and public service personnel. (Author/SJ) 
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PREFA CE 

On behalf of the Bureau of Library and Information Science Research, 
Rutgers The Stete University and the research team which has conducted 
the study, I am pleased to submit this report to f\ .1 A .-P ,l A . The 
report should be viewed as "interim" in that it covers only the first 
two of four phases. As originally submitted and agreed to by the U ^ .0 £ ., 
the proposal emphasized that each of the four phases was dependent on the 
results of each preceding phase. Phase IV was carefully detailed as the 
final stage for analysis and writing. 

However, since U.S.O.E. funds were not immediately available for 
Phases III and IV, the research team agreed to co:nplete a "final" report 
on Phases. I and II. Obviously, given the original proposal and subsequent 
work schedule, this decision curtailed any further efforts in Phase 11. 
Also, only partial analysis of the data collected in Phases I and II has 
been undertaken. 

Each of the 24 public libraries which have participated in the study 
werepromised compilations of the data they collected. Each library has 
received that information. However, follow-up with each library was 
planned as part of Phase III. Consequently, the research team has not 
been able to discuss the findings with the librarians, for purposes of 
clarification and modif ication--a serious limitation. On behalf of A.L.A. 
and the research team I want to express my deep gratitude for their active 
and enthusiastic participation which made the success of Phases I and II 
possible. 

A research project as complex and extensive as this one reouires the 
energy and thinking of many people. The key members of the research team 
are listed on the title page of the report. Specifically, without the 
contributions of Dr. Altman and Ellen Clark, the project could not have 
succeeded. 

The others who have shared their time and thinking with us include 
Ralph Blasingame, Jewel Walton, Henry Voos, Phil Clark, Kay McGinty, 
Harold Ray, Ben Weintraub and Mary Springman. To all the others, too 
numerous to mention, our appreci ation. 

Finally, I want to thank the members of our Advisory Committee whose 
encouragement and many suggestions proved extremely helpful, especially 
in keeping the project on course. To Dr. Frank Sessa, Chairman; Eleanor 
Ferguson, Walter Curley, Mary E.. Phillips, Phyllis I. Dalton, Harold 
Goldstein, John C. Frantz, our thanks. 

A special and personal debt of gratitude to Mr. Gerald M. Born, 
Executive Secretary of the Public Library Association, whose continued 
assistance allowed us to overcome many obstacles. 



Ernest R. DeProspo 
Project Director 
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INTRODUCTION 



The first two phases of the four phase study on The Measurement of 
Public Library Effectiveness have been completed. Specifically, these 
two phases were designed to (1) identify measurement criteria which would 
discriminate among public library services, (2) develop an operational 
methodology whereby local librarians could collect the necessary data for 
processing and refinement, and (3) demonstrate that the measurement criteria 
could be put into a theoretical model which would provide a professional 
basis for assessing the activity of basic library programs. 

'ihe notion of "measurement" is one which tends to exacerbate an 
dJready complex set of circumstances. The multiple views and long standing 
doubts over the meaning and use of this term often gets in the way of me- 
thodology. It is important, therefore, to state briefly the views of the 
research team on "measurement" and the values which were brought to the 
study. 

Too often numbers or statistics are treated as having an intrinsic 
scientific value. Techniques of measurement have been developed which 
are so subtle and so powerful that using them is often seen as all that 
is worthv/hile. Just as the overemphasis on definitions often approaches 
a faith in the power of the word, the mystique of quantity is often an 
exaggerated regard for the significance of measurement, just because it 
is quantitative, without regard for what has been measured or what can 
subsequently be done with the measure. What is really important is how 
one can learn more than he knows or how one can become more sure of what 
he thinks he already knows. Although measurement is by no means the only 
method of extending of solidifying our knowledge, clearly it has 'n impor- 
tant role to play in this understanding process. 

Measurement serves as a key device for standardization, a process 
through which we are convinced of equivalances among varied objects. As 
such, measurement allows more subtle discriminations and, correspondingly, 
more precise descriptions. Such quantitative specification allows us to 
bring our differing professional disagreements into a sharper and more 
illuminating focus. In the final analysis, however, it should be remem- 
bered that whether one can measure something is dependent on how one is 
able to conceptualize about the things to be measured. Conceptualization 
is dependent on our knowledge of the things to be measured and, most impor- 
tantly, on the skill and ingenuity which is brought to bear on the process. 

In a broader context, measurement is the delimitation and establish- 
ment of our ideas of thing. The Measurement of Public Library Effectiveness 
project needed to establish the appropriate limits, obviously, before 
becoming operational'. That is, initial decisions were needed to determine 
which elements of the public library program could/should be measured before 
the project team could conceptual ize them. Consequently, after many and 
various staff sessions, and aftertan extensive review of the existing lit- 
erature, the following "formula" was postulated as a representation in 



time: Document Availability, Staff Assistance, Physical Facilities and 
System Involvement minus Inconveniences constitute the key measurement 
criteria. 

The elements or factors incorporated in the "formula^' are similar 
to those utilized in other studies, Hov/ever, the approach is unique from 
any other measurement study undertaken in the way in which data is collected 
for each factor and in the way they are interrelated to produce a measure 
of effecti veii-^ss profile. Essentially, then, the project looks at effec- 
tiveness through three broad-based objectives of the public library: (1) 
materials availability, (2) staff availability, and (3) facilities usage. 
Against these three objectives we interact the user either through simula- 
tion ^r through actual observation. 

A Brief Overview of the Project 

This section briefly describes the four intended phases of the study. 
In addition, the specific rationale for the study is reviewed. A conment 
on the methodology is included with a concise cataloguing of the principal 
measurement criteria, or items, used. 

As originally conceive^, the study involved four distinct phases: 
Phase I- -Deve1opinq cri teri a^'which appear descriptive of the effectiveness 
of a public library program; development of a methodology for the process 
of data collection for the selected criteria; data collection on the 
selected criteria in a small number of pilot test libraries; and, establish- 
ment of tentative ranges of performance for each criterion. Phase 11 — 
Test of the criteria and methodology de*v<?loped during Phase I in a carefully 
selected sample of public libraries on a \iatiDnv/ide basis; preparation of 
a profile for each of the 'sample librariesV Phase IIi "-On-site visitation 
of some of the Phase 11 libraries to determi,:}e if the "measurement indicators 
developed from the refined criteria coincide v^ith professional judgment 
about the effectiveness of service provided by those libraries; and, in 
order to review and evaluate the profiles, a conference would be held, (Pre- 
liminary analysis of the criteria developed would provide the prime basis 
for the particular approach used in Phase III.) Phase IV—Completion of 
analysis and preparation of a final report. 

The basic rationale for this study was simply put in the proposal: 
''Attempts to measure the service capabilities of public libraries, either 
in relation to the needs of their communities or to the standards adopted 
by the profession, are hampered by the lack of criteria of quality or 
effectiveness," New measures are needed which "...must gauge effectiveness 
while eliminating extraneous factors. The required data must be such that 
library staffs can collect them with a reasonable expenditure of time. 
Some investigators have suggested such measures, but .have not taken the 
second step proposed here of matching the results against objective profes- 
sional judgements of effectiveness," 

A few caveats are in order. It is important to state what the study 
is not and to describe the considerations which have imposed constraints on 
the study. The especially strong "humanistic" strain which runs through 
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the profession. requires that open and honest consideration be given to the 
emotional and intellectual factors which approaches at "nieasurenient, " 
"evaluation," "quantification," etc., present. For mny librarians the 
doubt that one can infer quality from quantity is sufficiently strong 
that any method wh^ch asserts, per se , ' that numbers will provide the basis 
for professional consensus on what constitutes "acceptable levels of 
performance" runs the likely risk of outright rejection. 

Obviously, not all library activities are subject to reasonable quanti- 
fication or objective verification. Furthermore, numerous library activities 
take place which are considered valuable to "society" but not particularly 
as efficient or measureable as such, e.g., the time a librarian spends with 
a senior citizen. Indeed, the study to date shows more clearly than in the 
past writings what some of the limitations are in quantification. Thus, 
this study should be viewed as a first step effort at a fairly general or 
macro level at identifying some iniportant common core library activities 
which are amenable to quantification. Refinement which gets at detailed 
and specific analysis, or micro 1 evel .analysis , must await further study. 

The project has proceeded within the following constraints. First, 
library data necessary for the particular measurement tests developed 
must be collectable by the local librarian. Second, the measurement tests, 
while objectively based should be construed in a manner suitable for 
interpretation and action at the local level; that is, these tests are 
not to satisfy the interests solely of the researcher. While certain 
complex measurement .rocedures can be simplified beyond meaning, every 
effort has been made to develop measurement tests which are likely to 
have high priority in the public library and which are likely to be 
grasped quickly for purposes of understanding and action. 

With these considerations and constraints in mind, a number of criteria 
descriptive of public library programs were hypothesized after interviews, 
dialogue with the Advisory Committee to the project, and reviews of litera- 
ture on past efforts at statistical measurement. The following criteria, 
or "measurement indicators", have been refined and developed to date. The 
project personnel are highly confident that these criteria are descriptive 
and look forward to their analysis to determine the extent to which they 
are integrative; that is, to ascertain the degree to which one is or may 
be a function of the- others. 

The following is a description of the criteria studied and the aspects 
of these criteria that were- measured . The criteria differs substantial ly 
from the traditional data collected in that an attempt is made to relate 
systematically the various pieces which constitute a profile estimating 
the library's effectiveness of service to its users. 

I. Description of Collection: Materials availability? as tested 
through "BPR", Periodical, and Title availability sampling 
reduces the importance of the current practice of counting all 
the items for either external reporting or in-house tabulations. 
That is, utilizing samples of 500 items or less, the library is 
able to profile its materials within the context of user avail- 
ability. 
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• A. '^'BPR" Probability Samp1e--BasGd on the fact that there 
is a positive relationship between the recency of a 
collection and its use, .this measurement test was de- 
veloped to indicate the chances a user would have to 
obtain any book published in the last five years. 

B. Periodical Availabi lity .Saniple--This measurement test 
was developed to determine the chances that a user 
would have of finding an article cited in any one of ^ 
eight common indexes: Social Science and Humanities, 
Business Periodicals Ind ex, PAIS, Biological and 
Agricultural Index , Applied Science and Technology 
Index , Education Index , Art Inde x, and Readers Guide . 

'The test involves both the probability of ownership 
of a journal as well as the probability of obtaining 
a specific article. 

C. Title Availability Sample--This measurement test in- 
volved sampling the collection of a library in order 
to determine the probability a user would have of ob- 
taining any book owned by the library. In addition, 
this particular teSt provides data descriptive of the 
collection such as a breakdown by fiction and non-fic- 
tion, adult and juvenile, and so forth. 

II. Building Usage 

A. Description of Users--Users , in this case, were defined 
as those that actually come to the library. The descrip- 
tion categories are: 

1 . Sex 

2. Student-Nonstudent 

3. Grade Level of Students 

4. Occupation of Nonstudents 

B. Time User Entered and Left the Li brary--Thi s informa- 
tion was used to determine the pattern of use for the 
library facility such as average length of stay, high 
and low use time periods, and so forth. 

C. User Satisfaction--This measurement was designed to de- 
termine the satisfaction of actual users of the library 
based on their most recent experience in the library 

III. Circulation: In-Library Circulation is defined as those items 
which patrons actually sat down to read or examine, but which 
are not reflected in the normal circulation record. Outside 
Library Circulation is simply defined as any item(s) v/hich 
the patron has checked out of the library. 



A. In-Library Circulation--This measurement test was de- 
signed to determine the use made materials within 

. the library by the user. A breakdown is provided for 
both the type of materials used and the ' frequency of 
their use as well as the proportion of total in-li- 
brary circulation each type of material accounts for. 

B. Outside Library Circulation--This aspect of the study 
provides information on the number of items circulated, 
the number of users circulating books, the average num- 
ber of items circulated per user, and the percentage 

of users checking out books. 

IV. Facilities Usage 

This indicator is concerned with the use of equipment and 
special facilities within the library by its patrons. In- 
formation is provided on the highest incidence of use by 
item and the time of day when this occurred. These facil- 
ities include table seating, photocopier, meeting rooms, 
microfilm readers, and so forth, 

V. Patterns of Reference Usage 

A. Reference Activity by Time of Day--This aspect of the 
study provides information on the flow of reference 
activity during the day including peak and slow per- 
iods, 

B. Type of Question--A distinction is made between 
source related and directional questions and the 
number and percentage of each is determined. 

C. Source Related Questions-^Further information on 
source relat'^^d questions includes who asked the ques- 
tion ^ how it was asked, i.e., by phone or in person, 
who answered the question if it was answered, and tne 
source used to answer the question. 

VI. Public Service Personnel 

Public service personnel are defined as those employees of 
the library that have direct contact with the users. Data 
IS gathered to supply Information on their age, length of 
employment in the library, years of library experience, 
hours per v;eek at public service, sex, highest degree earned, 
and scheduling by hour of day. 

Existing Library Statistics 

The research team did not assume that existing library statistics 
are inadequate for purposes of measurement. An extensive effort was made 
to collect and analyze these statistics. The following account describes 
this effort. 
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State Library Statistical Gathering 

All state library agencies were asked to send copies of their latest 
public library statistical reports. Thirty-seven agencies responded. 
Preliminary analysis of these reports revealed the follovn'ng. 

From the reported data, forty-eight key statistical items on public 
libraries across the country were identified. Of these items, 50 percent 
or more states report on sixteen. These are: . . 

1. Name, address and telephone number of library 

2. County 

3. Name of library director 

4. Name of library system, where applicable 

5. Population served 

6. Hours open per week or per day 

7. Total number of volumes added to collection 

8. Total number of volumes at year end 

9. Total circulation figures 
10. Income from local sources 
11- Income from state sources 

12. Income from other sources 

13. Total figure of all income sources 

14. Expenditure on salaries 

15. Expenditure on library materials 

16. Tota! figure of all expenditures 
* \. * 

It can be seen that the most universal statistics shown in state 

reports cover basic facts about public libraries such as book stock, 
circulation, income and expenditures. While useful, they remain at 
best generalized reports directed towards tte uninitiated lay public, and 
offer little direct evidence of effective service to the patron. 

Outside o-^" these sixteen basic items, some states break down these 
items into types of material held (New York), various sources of income 
(Pennsylvania), and attempt at providing unit costs of certain services 
(Utah). Beyond this breakdown the individual reports range freely, provi- 
ding anything from the year of foundation (Colorado and Indiana) to numbers 
of reference transactions (California and South Carolina). 

A few states endeavor to put 'life' into these statistics by reporting 
on the use made by the public of a service provided. California, Pennsyl- 
vania, and South Carolina, for example, provide figures on inter-library 
loans. 

Iowa and Minnesota show the percentage of population unserved by 
libraries, although it is not clear if the figures relate to the lack of 
a library in a given area or, more important, that a percentage of the 
population does not avail itself of library service. 

Kansas breaks down the total circulation figures by patron/schools/ 
( hospitals/ other institutions loans, which make these figures more mean- 

^ ingful. Indiana and Tennessee give details of bookmobile service showing 

the number of stops and breaking these down into kinds of patrons served.- 
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Some states provide a breakdown of audio-visual materials held, but 
neglect to provide circulation figures for these non-book items. 

Finally, only nine states report on the numbers of professionals 
and nonprofessionals serving the patron. 

U.S.O.E. Library Statistics 

f:;;tensive analysis of library statistics published by the United 
States Office of Education was undertaken by the research team.^ A separ- 
ate reporL details the results of that analysis. The basic conclusion^ 
is that the "data categories presently reported are insufficient and 
incomplete nieasures of library effectiveness..." 

Public Library Questionnaire Survey 

In September 1971, questionnaires v/ere sent to a stratified random 
sample of two hundred and fifty-four public libraries in the United. 
States. The sample, drawn from the Ameri can Library Director y, was 
stratified primarily in terms of size." Since there is considerable 
variation in the size of libraries, this variable had to be taken into 
consideration in any attempt to measure effectiveness. Libraries with 
greater resources might be able to offer a broader range of programs, 
specialized services and the like; elements that may be lacking in other 
libraries due to their more limited resources, for example. In such a 
case, comparison v/ould be misleading-.-; Thus, since all earlier research 
has concluded that the most .rel iable factor for determining size is 
budget, libraries in this sample were classified in terms of size by the 
following budget categories: 

Small Libraries $100,000 - $249,999 

r^t^ Medium Libraries $250,000 - $749,999 

Large Libraries $750,000 - $3,499,999 

Given the limitations of time and money, it was decided that both the 
very small and the very large public library would not be included within 
the universe of public libraries at this point and throughout the study. 
Secondarily, there v/as also an attempt to obtain an equitable geographical 
distribution within the sample. 

The one hundred and tv/enty-four respondents to this questionnaire 
answered eighty-five questions on the ease or difficulty that their 
library v/ould have in providing different types of information for the 
last five years. Each library ms asked to respond in the following way 
to each of the items : 

I have the exact figure (s) 
requested or a very similar 
breakdown in my files and 
records 

I do not have the exact fig- 
ure($) on hand, but my files 
or records contain the infor- 
mation which v/ould allow me 
to compute the requested fig- 
ure (s). 



Easy to Provide 



U Difficult to Provide 

EKLC 
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I Impossible to Pro- J have neither the exact fig- 

vide urG(s) on hand nor the files 

or records to compute the 
necessary figure(s), I would 
be required to set up a new 
record-keeping system or do a 
special study in order to pro- 
vide the information. 

C -- Detailed Records The information is currently 

are not Retained collected but a record for 



N Not Applicable 



past years is not retained. 



On the basis of the responsis to the information questions, an analy- 
sis of the relative availability of different types of information was 
made. The following provides a breakdown on the eighty-five items of 
information for all public libraries responding as well as for each public 
library size: 

Availability of Library Statistlcs-1965-1970 



( 



(Bases) 


.Total 


Large 


Medium 


Smal 1 




(124) 


(34) 


(43) 


(47) 


Total square feet of building 


98% 


1005^ . 


98% 


98"^ 


Budgeted amount for equipment/supplies 


98 


97 


100 


96 


Total number of volumes in collection 


98 


97 


98 


100 


Total number of volumes added yearly 


98 


97 


98 


98 


Total number of volumes v/ithdrawn 


98 


94 


100 


98 


Annual per capita expenditure 


97 


97 


98 


96 


Total number of people in service area 


96 


100 


95 


94 


Budgeted amount for professional salaries 


94 


94 


98 


89 


Budgeted amount-nonprofessional salaries 


94 


94 


98 


89 


Total square feet of stack area 


94 


94 


93 


94 


Total square feet of reading room 


94 


91 


95 


Q6 


Budgeted amount-building maintenance 


93 


94 


95 


89 


Budgeted amount - print resources 


90" 


94 


91 


87 


Total number of phonograph records 


89 


94 


88 


85 


Budgeted amount - nonprint resources 


87 


94 


86 


83 


Total number of items circulated yearly 


87 


91 


86 


85 


Number ILL items borrowed annually 


86 


91 


86 


83 


Number ILL items loaned annually 


82 


85 


91 


72 


Total number of microforms in collection 


82 


8? 


86 


79- 


Total number of periodical titles 


82 


82 


74 


89 


Number of registered borrov/ers (total) 


81 


71 


81 


87 


Total number of other A-V materials 


79 


79 


65 


58 


Hours of staff time on building mainten. 


78 


79 


81 


75 


Total number juvenile fiction volumes 


78 


74 


81 


79 


Hours of staff time on technical services 


77 


79 


81 


72 


Hours of staff time on circulation 


77 


79 


79 


72 


Hours of staff time on administration 


76 


76 


81 


70 
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(Bases) 


Total 


Large 


h]ed i um 


Snial 1 




124 




(43) 


(47) 


Hours of staff time on reference 


72% 


7A% 


11% 


10% 


Total number juvenile nonfiction volumes 


73 


68 


75 


77 


Total nuinb^r of films in collection 


71 


88 


70 


60 


Number of juvenile registered borrowers 


71 


65 


68 


79 


Annual per-regi ste red -borrower expend i ture 


71 


59 


74 


77 


Number of adult registered borrowers 


70 


62 


68 


79 


Total adult/young adult non-fiction vols. 


69 


62 


77 * • 


68 


Total adult/young adult fiction volumes 


69 


59 


79 


66 


Total number of reference volumes 


68 


68 


67 


68 


Percent *time spent at reference desk 


68 


59 


74 


68 


Circulation of audio visual materials 


66 


82 


65 


55 


Hours staff time on org. and main, of coll. 


65 


65 


65 


64 


Hours staff time on public relations 


64 


71 


58 


64 


Total number of periodical volumes 


61 


62 


65 


58 


Percent time spent selecting books 


57 


41 


60 


64 


Total number of titles added yearly 


56 


85 


61 


32 


Total number of filmstrips in collection 


53 


56 


58 


47 


Total number of volumes lost yearly 


50 


50 


58 


43 


Circulation of adult fiction 


50 


44 


47 


57 


Circulation of adul t nonf i cti on 


50 


44 


47 


57 


Number of ILL requests you did not satisfy 


49 


47 


49 


51 


Circulation of juvenile fiction. 


48 


44 


4? 


00 


Budgeted amount - technical services 


46 


51 


51 


34 


Circulation of juvenile nonfiction 


46 


41 


42 


53 


Budgeted amount for admini stration 


44 


53 


51 


30 


Amount received to provide system services 


41 


50 


49 


28 


Budgeted amount for circulation 


40 


53 


47 


26 


Amount spent for system services 


39 


35 


49 


32 


Circulation for periodicals 


38 


29 


32 


49 


Total number of titles v/ithdrawn yearly 


36 


29 


49 


28 


Budgeted amount for reference services 


35 


44 


40 


23 


Hours of staff time on system activities 


34 


44 


35 


21 


Total number of titles in collection 


34 


38 


42 




Number requests sent via teletransmission 


33 


47 


35 


21 


Budgeted amount - other reader services 


33 


44 


37 


21 


Number requests recVd via teletransmission 


32 


47 


33 


21 


Number reference questions referred elsewh. 


32 


27 


33 


36 


Total number of reference titles 


32 


21 


37 


36 


Total number. of government documents 


31 


50 


33 


17 


Total number adult/young adult fict. titles 


31 


32 


42 


21 


Total no. adult/young adult nonfic. titles 


31 


32 


40 


23 


Total number juvenile fiction titles 


31 


24 


47 


23 


Total number juvonile nonfiction titles 


29 


21 


42 


23 


Number ref. requests you ans. for other- 


28 


27 


28 


30 


Number items lent to universal borrowers 


24 


18 


30 


23 


Number patrons enter building daily 


23 


35 


21 


15 


Budgeted amount for (unspecified) other . 


23 


21 


26 


23 


Total number of titles lost yearly 


23 


-18 


35 


17 


Number ref. questions referred/not ans. 


22 


12 


21 


30 


Budgeted amount for system membership 


17 


12 


26 
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The Usefulness of Public Library Statistics 

Librarians in the national sample v;ere asked what measures of 
quality 3 which could be statistically determined, would they like to 
have available for use in evaluating their services. As one might expect, 
a variety of responses v/ere reported by the librarians. However, the 
resp.onses, once categorized, depicted a visible trend-^ more information 
directly related to the user . 

Basically, the suggestions centered around user satisfaction and 
user activity. Thus, for example, one librarian suggested the following: 
"A record of individual patron satisfaction and dissatisfaction for every 
( service transaction between him and the library. A rating scale for 

numbers of people in th-? service area reached in some way be library 
service compared to other agency ratings. (This comparison could also 
be made with other libraries using the same criteria.)" Another librarian 
noted: "Judgment of each person who uses the library during the year as 
to how well he was served by it and a report from him on areas where he 
was not served well." Another librarian said: "Mainly, some measure of 
patrons' reaction or a statement as to their satisfaction with the lib- 
rary's response or service." 

Intuitively, the librarians were saying that the data currently 
collected is not people or user oriented. The decision to simulate the 
user through the methodology of probability statistics a^ key elements 
in the measurement criteria developed was reinforced by these responses. 
The data collection approach, as illustrated in the Instructional Manual 
for the Collection of Selected Public Library Information , sharply demon- 
strates the efforts of the research team to fill the user-oriented informa 
tion gap. 

The same 1 i brarians were asked to indicate which statistics could be 
reported which v/ould accurately reflect their library's effectiveness. 
Again, the responses centered around the user, as contrasted to "things". 
The following observations made by these librarians illustrate this 
point: 

( 



ERIC 



12 



"Written or verbal testimonies of pGople--nuin- 
bers do not show effectiveness," 

"None, unless vie place a counter on the; door to 
count the nuinber of patrons each day." 

"Number of people who went away dissatisfied (or 
unsatisfied)". 

f 

^'Number of people uning the library--not just 
circulation--especial ly in special collections. 
Requests not filled. What percentage of pa- 
trons are not finding what they want." 

"Gallop-type polling of representative samples 
of the public at different times might do it. 
Certainly circulation figures and borrowers 
regi stered don * t. " ' 

Of course, some librarians responded by noting that they "doubt that 
such statistics exist" and "you tell me." 

These librarians were also asked to cominent on the use of stat- 
istics to measure effectiveness. From the responses received, it is 
not difficult to see how doubtful they are that one can measure effec- 
tiveness statistically. Ref resentati ve comnents were: 

"Statistics do not evaluate quality of service 
or degree of satisfaction of community and/or 
patron needs. I know no way this can be mea- 
sured statistically." 

"Statistics as kept do not describe the quality/ 
nature of individual servi ce--only the voluine 
of it. I don't think stati sti cs--numbers--or 
non-verbal written data can describe the human- 
istic situation in the subjective atmosphere in 
which it takes pi ace. " 

"Statistics can be misleading and none can give 
a complete picture of library service. I abhor 
the time involved in too many "statistical reports' 
and do wish a uniform system could be devised 
that would satisfy all types of libraries-- 
don't you." 

"Statistics do not and I don't believe could 
ever include the human factor in giving ser- 
vice which I believe to be the key. I do not 
think that it is possible to reflect statisti- 
cally the effectiveness of a devoted reference 
librarian or a children's librarian unless a 



13 



person was asked to fill out a brief question- 
naire each time he used the library. Business- 
men often do not need help and know how to use 
the services and directories. It would be dif- 
ficult to measure the library's effectiveness 
to this extent without invading privacy." 

The above observations fairly accurately depict the general kinds 
of reservations or doubts which public librarians possess when the' 
question of statistically measuring effectiveness of service is raised. 
However, when all of their responses are viewed together, it is also 
clear that public librarians need and want better and more appropriate 
ways of "measuring" the services they offer through some kind of user 
orientation rather than the current "thing-oriented" approach. Phases I 
and II of this study are clearly consistent with this view and expressed 
need. 

Pretest, Pilot Libraries and Instructional Manual 

The methodology for the study was developed and pre~tested in four 
pilot libraries during Phase I. Following innumerable staff sessions, 
dialogue with library educators and practicing librarians, and prelimin- 
ary analysis of the results of the literature search, the decision was 
made to select three public libraries of "like character", i.e., similar 
in terms of budget , size of collection, number of branches, number of 
employees, and so forth* The results of the pilot would provide a basis 
to make initial judgments on both the forms used to collect the date as 
well as with the sufficiency of . the data itself. In order to insure ex- 
posure to as great a variety of variables as possible, it was decided 
that medium to large sized libraries should be selected, 

The project staff proceeded to develop the method and forms for 
data collection and then to the actual collection of the data in the 
pilot libraries. In this way the research team could test the forms 
developed, including the likely problems which any data gathering ap- 
proach might present to the local librarian, as well as to determine 
the time required to collect the information. One of the three pilot 
libraries was generous enough to provide a member of its staff as an on- 
going resource to the study and to work as a member of the research team. 
Consequently, we were able to have a very effective link for feedback on 
the methodology as an integral part of the research operation. 

With the completion of the data gathering in the three pilot librari 
the Instructional Manual was drafted. We then decided to select one more 
library to test the Manual, that is, to see if the local librarian could 
collect the information within the limits established. It was assumed 
that a key element in the data collection is the existence of sufficient 
staff. Consequently 5 the greatest problem might very well be with the 
"small" public library. As a result, the director of a small library 
was asked to participate in the study. A special one day workshop on 
data collection procedures, following the instructions proveded in the 
Manual, was conducted for the director by the research team. 
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The major goals of the pretest, then, v;ere to see if (1) data could 
be collected on each criterion or item, (2) could an average library 
collect the data on its own with minimum supervision, and (3) was each 
item discriminatory, i.e., did it show individually or in combination 
with other items differences in the four libraries which professionally 
trained librarians could observe in. a gross sense or surmise from 
interviewing employees. It was assumed that most significant criteria 
or items had an outward manifestation in the form of some kind of quanti- 
fiable action or statistics. This assumption was made for research 
purposes and does not mean that the project personnel believe that all 
aspects of library programs can be or should be translated into statistics 

Techniques 

The following is the list of items selected as the measurement 
indicators. Analysis of the pilot library data indicate that all but 
one, outside library circulation^ significantly discriminated among 
libraries. The items are: 

. Title Avai 1 abi 1 i ty 
"BPR" Availability 
Periodical Availability 
In-Library Circulation 
Building Usage 
Patterns of Reference Usage 
Public Service Analysis 
User Facilities'and Equipment 
Outside Library Circulation 

All items are either based on probability statistics (utilizing 
random sampling) in which a user simulation covering a variety of library 
service exchanges is employed, e.g., estimating the probability of any 
user obtaining a book published in the last five years from the library, 
or on total factors, e.g., extensive information on all the users enter- 
ing the- 1 ibrary at selected time intervals. 

For the latter consideration, the research t^^am was concerned with 
testing such relationships as (1) the time the user spent in the library 
with the number of items checked out, (2) the tinje spent in the library 
with the extent of staff assistance, (3) the time spent in the library 
with user satisfaction, (4) the ratio of in-library use with outside circu- 
lation, particularly as that ratio might be related to physical plant, 
collection and staff. 

Summary of Pilot Library Data 

The complete data distribution (less those factors inappropriate 
for Phases I and II, e.g., community outreach) are included in the 
appendix of this report. It was assumed that the four test libraries 
were, in fact, sufficiently different from one another that the measure- 
ment data should reveal some of the differences. Preliminary analysis 
of the pilot data (see appendix to the narrative) clearly reveals some 
of the differences and reinforced the conclusion that the criteria v/ould 
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indeed discriminate from library to library as v/ell as from size to 
size. 

The data from the pilot libraries have been processed and 
analyzed extensively, although not completely. On the basis of this 
analysis, the fol (owing appear to be effective discriminators among 
libraries: 

1. Building Use Tfr , "^^.raries • ^ 

varied significantly uii such factors as 
occupation of patrons, grade level of 
student patrons, proportion of student 
patrons, length of stay, and time of day 
v/hen use is highest. 



Title Availability The pretest li- 
braries vary significantly on the avail- 
ability of titles listed in their shelf 
list. There is also considerable varia-- 
bility in the age of collections. 



3-. Peri odircaT Avail ability The availa- 
bility of periodical articles varied 
systematically among pilot libraries. 

4. Reference Activity The pretest li- 
braries varied considerably on the pro- 
portion of actual reference questions to 
directional questions. Also, there v/as 
variation in the proportion of questions 
received by phone as opposed to in per- 
son. 

5*. In-Library Circulation -- The volume and 
variety of materials used within the li- 
brary by patrons varied significantly 
among the pilot libraries. 

The research team is convinced that with the analysis of the full 
data base, the following tv/o discriminators will further differentiate 
public library services. 



1. Characteristics of Reference Person- 
nel At this point the sample is 
too small to draw any conclusions. 

2. Organizational Health (Likert Scales) 

Although the pretest libraries vary 
on this dimension, more extensive and 
sophisticated analysis will have to 
be carried out in order to determine 
the exact nature of this variation. 
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3, '^BPR" Availability More data is needed 
in this area. 

The following areas have proved to be prohlenati^ : 

1 Co^'v ^ ^.ach and Library Sponsored 

programs; and, 

2. Library Information Questionnaire 

This instruments designed to collect 
tradi ti onal statistical informati on 
about the participant libraries over 
the last five years , has produced 
random as well as, in mny cases, in- 
complete and incomparable data. Be- 
fore this aspect of library service 
can be measured adequately and com- 
pared to other measures, some form of 
systematic reporting, and also cost 
accounting, v^ill have to be developed. 
The LIBGIS system might be a comple- 
mentary and viable alternative to 
this aspect of the study. 

PHASE II DATA COLLECTION 

The. data collected in the four pilot libraries was refine! and 
analyzed before a:ny major decisions v/ere made in Phase 11, Ths' results 
of the pilot data analysis were such that the project personn^i were 
satisfied that the Phase II undertaking would be viable, Natisrailly, 
the critical factor was v/hether or not the measurement indicators 
selected did in fact discriminate among the services offered b different 
libraries. 

Tihe selection of Phase II libraries, which finally totale.? twenty 
was infTuenced by a number of factors. The most important weirr-- (1) 
geographical spread, (2) inclusion of public libraries to fall within 
fout siBall, medium and Targe size categories, and (3) a commi tni,Virit to 
the overall objectives of the project by the library director. Data 
collection was planned to cover a two week period. Roughly fnTit5/-f ive 
hours of time; was required to collect the desired information xh:.ring 
week of the sludy. A coninitment of three full days were required in 
week two. In addition, each participating library was asked to appoint 
a project coordinator to supervise the data collection and forward the 
iresults to the Research Bureau at Rutgers, In order to approxl/ate 
"normal" conditions, the libraries participating in the study w^re asked' 
to give no prior publicity to the project. 

Three regional workshops were provided for the project coordinators 
appointed by the libraries participating in Phase II. The first work- 
shiQ)p,v/as giver in New Brunswick, New Jersey; the second in Atlanta, 
GeiDrgia and t ;e third in San Francisco, California. No particular 
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problems developed during any of the v/or r, shops . The librarians were 
'^le to readily grasp the data gathering instructions. 

The data collected in Phase II have been submitted to preliminary 
analysis and profiles have been developed for each of the participating 
libraries. 

LIMITATIONS OF. PHASES I AND JI 

Since- interpretation and data analysis leading toward data refine- 
ment and specification of the theoretical model are the prime activities 
for Phases III and IV of the study, only descriptive information is 
provided in this interim report on Phases I and II. Consequently, 
important areas in which data have been collected could not be covered 
in this report. The following data areas represent the most significant 
factors which must await analysis: 

Community Outreach and Library Sponsored Programs 

All Phase II libraries supplied extensive data on their community 
outreach activities and other library sponsored programs. This informa- 
tion does not lend itself to any convenient typology and therefore can- 
not be simply described. Our analysis will aim to discover to what 
extent, if any, a relationship exists between the measurement indicators 
for the library and the kind of outreach and/or library sponsored pro- 
gram activity which take place. 

Library Information Questionnaire 

Each Phase II library was requested to supply extensive library 
statistics covering the following areas: book holdings, documents, 
periodicals, A-V materials, users,, circulation, allocation of staff 
time, allocation of budget, physical plant, and system activities. In- 
formation on all these areas was asked for a five year period (1965-1970) 
to give a basis for comparison. It was not expected that the libraries 
would be able to supply all of the information requested. They were 
asked to supply the information from the library statistics that they 
were already collecting. The data from this questionnaire will be analyz- 
ed in Phase III to see what relationship, if any, exists with the other 
measurement indicators. 

Staff Questionnaire: Likert Scale on "Organizational Health" 

This questionnaire was adapted from The Human Organization by Rensis 
Likert and used with the permission of the McGraw-Hill Book Company. The 
purpose of the questionnaire is to see how the library operates organiza- 
tionally. Every employee in the library, excluding volunteers, were 
asked to fill out the questionnaire. It is hypothesized that the way 
in v.^hich the librarian views the organization will influence the services 
provided. Obviously, the k'ind of data supplied is non-descriptive and 
must be analyzed in conjunction with the measurement indicators developed. 
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Final! V, various data areas in which great detail is provided, 
e.g., brearr-,own and use of collections, are not included as part of 
Phases I and II. Again, only detailed analysis will allow the research 
team to decide which data is essential. 

CONCLUSION 

As a result of the first two phases of the study, the following * 
considerations are clear. First, the methodology developed is ^appropri- 
ate to the overall objective of the study, i.e., selected data which 
measures various aspects of the public library program can be collected, 
wi'th minimal assistance, at the local library level. Second, the data 
itself does discriminate the performance of one public library from 
that of another. Thirdly, the data comes much closer than present 
library statistics to meeting the demands of both the librarian and the 
patron for "user-oriented" indicators which are necessary if the public 
library is going to reflect accurately the variety of activities that 
it is undertaking. 

Each of the twenty libraries which have participated in this study 
will receive a "mi ni -management report" which will tell them v/here they 
stand on each measurement item selected at this point in the study. In 
addition, they will be able to compare their performance against each 
of the other nineteen libraries. However, until the study is completed, 
direct follow-up with each library for purposes of explanation and re- 
finement will not be possible. Also, there will be no opportunity for 
these professional librarians to meet as a group and express their over- 
all judgments on each measurement item, an input which is cl^cirly manda- 
tory if the approach selected as a result of this study is to find 
general acceptance in the profession. 

The data to be tested as a result of the study do far (which the 
U.S.O.E. has already received) offers several advantages over the various 
measurement systems used by the states: 

1. The data is comparable on a regional, state, 
or national basis as to (a) type of major 
services, (b) quality factors which modify 
quantitative items, and (c) personnel and 
management quality; 

2. Not all items need to be collected each 
year in order to maintain the integrity 
of the total system; 

3.. A profile can be set. for each library. At 
the same time, special local factors can 
be added for local decision-making; 

4. The data is in a form to facilitate 
setting up a national data bank; 



The data can collected locally with 
a niininium of t'r.e ;uid supervision from 
a state or federa*! agency. 

Sampling techniques are used to measure 
activities otherwise not considered as 
subject to iiieasurement. 

Longitudinal reviews over a period of 
time, e.g., half a decade to a decade, 
are f aci 1 i tated . 

A factor for evaluation of library nian- 
agement is included. 
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INTRODUCTION 



u'^^ Libraries are a major public service whose total impact and cost arp 
hidden from direct public view because of their diversUy as schoor pub- 
lic university, private research, institutional, and special lib?aHes 
Hubiic libraries alone had a total income in 1970 in excess of $700 OOo"oon 1 
Since the growth in income for nearly two decades .has been steady and has 
a^ph s?L^^'na^^^ I'r r \PP-°r-tions, and since the LLi Tocu" 
?han a^P .vL-?.Mo modern American society require more resources 
benefit, ^rri h nn ^ ^^.^^ increasing public concern about the 

y^T Va jccrumg from public library service. The point has now been 
reached where traditional arguments for public support of expanded service 

?^onaf re'snnnL^^'^^f^i 'I P^^^' d° ^'oke as s^fon n m - 

tiona I response. Indeed, a negative reaction could develop raoidlv if 
by accident, improper or poor use of funds occurred in several ocalUies 
or If strong popular pressures demanded a higher priority for cena n 
other socia services. This position is well staged as a general theme 
by Suchman in Evaluative Research .^ ytinerdi zneme 

ever to acJpnl^ thp'^^fn sophisticated' publ ic is less willing than 

^nniw ?^ P?-^^? need for a community service on 'faith alone. Increas- 
arfJ: P"'^^^'^ 'I demanding proof of the effectiveness of various pro- 
grams. The current desire to judge the value of social institutions is 
only one aspect of society's belief that many social problem can Ee met 
most effectively through planned action based upon existing knSwledae 

ih"e p M?c' xo ctj'^-^' ^^"f ,^°^^tions in step' with'"d'Jan'c^n7knoS edge. 
Ihe public expects bigger and better services. Such services are becoraina 
defined more as public rights than individual privileges " becoming 

.•F ^"^ without exception measures of effectiveness 

of social programs have not been formulated as rapidly or in ^he See 

with'a naradn^ f.^^i .1 "'^^ inadequate analytical tools, but rather 
and p?f?^- ? ^^^i ^^J^^ ^^^"^^9^ citizen. As an individual, effective 
?n .n^h °^ P"f^^'= abstract concept which he supports 

detirm na??on S tT/'l- '''''^ .''^ ^'^^'^ discussions^^ Howeve?! S a 
rp.?w !• 2^ effectiveness in a specific program requires a significant 

nl? s tale a°Ihm°"''h'' negati vely'becaSse ? y " 

either e?Jpr^-vpnlL /'l.^''^'°"'^ which are not a function of 

t on witJ mPnL? . ^^^^^^^^P^: ^or example, his personal associa- 

benef ts fronf^LSd'^^M-'^.P;?'^"^" ^^'"^ ^^^"^ accepting an argument that 
fo^ a DrPvIntnu ^r^''' ^^7^ libraries exceed those 

Tor a preventive high school drug program. 

Theoreticians of organizational behavior have recognized this conflict 
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and have been interested in defining effectiveness for at least two 
decades. For the most part, they have worked from a premise that effec- 
tiveness is synonomous with the attainment of an orgarii2ation ' s objective 
and the continued legitimacy and viability of the organization itself 
Although there is a certain value in this kind of definition, Etzioni 
pointed out a decade ago that while the stated goals of an agency serve 
as Clues, to actual goals they cannot be accepted always at face value. 
Goals are symbols or ideals which are more attractive than the reality 
which the organization attains. Thus, he argues, organizations can al- 
most always be judged ineffective (and hence in need^of more support). 
He favors examining how the organization allocates its resources and 
directs its efforts to ascertain real goals. 3 

In later organizational theory, a wide variety of ideas have been 
advanced about ways to measure quality, efficiency, responsiveness, or 
effectiveness, these being illustrative terms used by different authors. 
Some techniques of measurement have been adopted under the pressure to ' 
make public decisions in an objective (non-political) manner. Nearly 
every proposal or technique has been subjected to severe theoretical 
criticisms, particularly by social service oriented people as contrasted 
with the mathematical economist and statistician. For example, Werner 
Hirsch concludes in a well known article on the quality of government 
service that ". . . it is apparent that efforts to measure the quality 
of urban government services offer exciting challenges and prospects. 
Much more work is needed along three lines of inquiry--defining service 
umts in real terms, identifying their major quality characteristics, 
and estimating the money value and money-cost of these characteristics."'^ 
Aaron Wildavsky commented in 1966 about cost benefit and system analysis: 

Studies based on efficiency criteria are much needed .... 
My quarrel is not with them as such, at all. I have been 
concerned that a single value, hov/ever important, could 
triumph over other values without explicit consideration 
being given to these others. I would feel much better if 
political rationality were being pursued with the same 
vigor and capability as economic efficiency. In that 
case, I would have fewer qualms about extending efficiency 
studies into the decision making apparatus. 5 

In an earlier paragraph, he quotes one economist as saying: "One can 
view cost-benefit analysis as any thing from an infallible means of 
reaching the new Utopia to a waste of resources in attempting to measure 
the unmeasurable." 

The problem of measuring effectiveness in the modern corporation 
IS no less easy in fact than iii the public service. The literature in 
this field recognizes that there are no simple techniques or agreed on 
objectives. The amount of profits are recognized in this era of social 
responsibility of business as not necessarily the only or best guide of 
effectiveness; and cutting costs is admitted as not desirable in all 
cases in either the short or long run. Research and development, staff, 
public reiations, and planning are units which are particularly difficult 
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to evaluate. Wessel and'Cohen show this new thinking in part in their 
1967 note about 62 techniques used in business to study performance .6 

In the literature on social services in general, the same diversity 
in ideas and hopes exists. In fact the body of literature is so large 
that several universities offer new specialized degrees in plannino and 
evaluation. As a prelude to the next sections on library measurements, 
it can be noted here that for social services in general, a wide variety 
of tools are used ranging from standardized tests of achievement to 
preset_ standards of operation based on a norm of current practices. 
In addition, at all levels of government a large body of statistical 
data is collected. Although the data have been refined in recent years, 
for the most part, though they are still descriptive, are not precise, 
and measure the intangibles of human development only indirectly bv assuming 
certain relationships, e^. , equating average size of classroom or^ average 
number of beds or minimum educational attainment for employees with a set 
quality or level of achievement. Resources, consequently, tend to be added 
to reach these averages, and by definition, the achievement is reached. 
Obvious defects or deficiencies are then usually ascribed to other external 
social factors or as acceptable variations in an imperfect system. 

The theoretical statements of effectiveness of social services tend 
to stress a) what is hoped for, b) models for which it if admitted there 
is insufficient data to make them operable, and c) conflicts of personal 
values. Moreover, the complexity of life, i^. , added determinants of 
social and intellectual behavior, has increased faster than our ability to 
isolate the determinants and measure them as discrete units. The apparent 
lack of success to develop measures of effectiveness, therefore, is not the 
fault of any of the professions. It is in this broad social effort that 
the library profession has attempted consistently for three decades to 
perfect a system which would inform the public and profession about the 
nature and quality of service being returned for the social investment 
in public libraries. 



Objectives of Public Libraries 

The notion that a library's effectiveness can be judged only in re- 
lation to the objectives which it pursues recurs so often in the literature 
that it has become a cliche. However, there are serious problems in using 
stated objectives as a yardstick to measure effectiveness. Many libraries 
have not adopted specific objectives. In fact, failure to formulate ob- 
jectives was cited by Martin as the sixth most critical problem facinq 
public libraries, 7 

All too often, the objectives of the library as they have been stated 
in the professional literature are so vague that they are meaninaless. 
Examples are: human understanding, civic enlightenment, personal develop- 
ment, community development, creative and spiritual development, and the 
old standbys of information, recreation, and education. 



Are all of these objectives of equal importance? Should they only 
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apply to the self-selected user of the community at large, or both? More 
importantly, hov/ can one identify the library's impact on "civic enlighten- 
ment," or "personal development" from the impact of other social institu- 
tions in the community? None of these objectives is operational in terms 
of indicating the desired outcome because there are no criteria to assess 
performance. 

Perhaps the profession has been unable to agree on objectives because: 

1. There is no well accepted theoretical base of philosophy of ser- 
vice accepted by the library profession. 

2. The components of good library service have never been defined 
except in very general terms. 

3. The question of who should set objectives for public institutions 
has never been settled. 

4. The "library's public" is assumed to be composed of many different 
and conflicting Interest groups. Whose interests shall have 
priori ty?o 



Standards and Statistics 

_ _ Even though it 1s frequently argued that measurement of performance 
IS impossible in the absence of clearly defined goals, the profession has 
nevertheless promulgated a series of standards expressly designed to set a 
P'^'io'""' the adequacy of public library programs. The intent is clearly re~ 
fleeted in the following statements taken from the 1943, 1956, and 1966 
public library standards. 

1943 Standards: The major purpose of this study is the formulation 
of working standards for public library service which may be used 
as a measuring Instrument of the adequacy and efficiency of present 
library service . . . 

Standards should be used as a national measuring stick ... .10 

It is expected that the standards will also be used by state and city 
planning and other officials, by library surveyors, library boards, 
and other interested groups in evaluating the library service of 
individual cities, counties, and states 

1956 Standards: This document provides a guide for the evaluation 
of public library service. It is intended for the use of librarians, 
library boards, government officials, and Interested citizens in 
assessing the adequacy of their present library services and in 
formulating plans for Improvement 12 

The present statement must be understood as a guide to total evalua- 
tion of public library service . ; . . .13 

o 
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1966 Standards: This publication used phraseology almost identical 
to the two statements quoted above from the 1955 publication."''^ In 
addition the document stated: 

"Only such standards have been included as have ? dirs"t and positive 
relationship to quality of library facilities c serv:ce."15 

; 1 three publications define the word standard as a specific cri- 
.iL.rion against which cxisquacy and quality can be tes"'5d and measured. 

Standards have traditionally been adopted to set a "base line of 
niodern public library service"l6 and to "define minimum adequacy of 
library facilities"'/ in quantitative terms of book sto'ck. finance, 
buildings and staff. 

Initially, Minimum Standards for Public Library Systems , 1Q66, de- 
emphasized quantitative factors and stressed what were called "quidinq 
principals." 

In response to a feeling on the part of some librarians that' 
Minimum Standards for Public Library Systems. 1966 . is sometimes 
too general to be useful, the Standards Committee of the Public 
Library Association devised certain Statistical Standards, which 
were approved by the membership in June 1967. These are inter- 
pretations in mathematical terms of specific standards, intended 
for use with the more general standards, and are now included as 
a measure of document. "18 

The profession's acceptance of quantitative standards as a measure 
of library performance implies that quality is inextricably bound to quan- 
tity. Therefore, the descriptive statistics reported by individual 
libraries for such items as expenditures, staff and holdings, etc., are 
by implication meaningful indices for evaluating overall activities of 
libraries. 

On the other hand, there has been considerable discussion in the 
literature that recorded statistics cannot be used as indicators of li- 
brary effectiveness. The main arguments advanced by most of those who 
have taken up the pen. in this fray are: 1) that the statistics are in- 
herently unreliable, 2) that quantity cannot always be equated with quality 
and 6) that present statistical categories measure resources, not actual 
service. 

The charge that reported statistics are unreliable has considerable 
merit. The problem is that not all libraries count the same items in the 
same way. For example, how are volumes counted--by physical or biblio- 
graphical unit? Are serials included in the volume totals? Are aovern- 
ment documents counted as volumes? Are microfilms counted by the^reel or 
by the bibliographical volume? Are bound pamphlets counted as volumes? 
What makes a librarian a professional--a degree, experience only or simply 
a job title? 'The counting problem is compounded by the fact that state 
library agencies officially define the terms differently. 
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This lack of a uniform statistical reporting system has hindered com- 
parisons of library statiitics and given anmunition to the anti-numerical 
faction. In 1966, the American Library Association attempted to clarify 
■tounting procedures by publishing a handbook outlining how library items 
should be counted or defined.'-^ In the six intervening years no follow- 
up studies have been reported in Library Literature as to the acceptance 
of the concepts presented in this manual or its effect on present re- 
porting. . 

That quantity does not always equal quality has been the classic 
argument in disparaging the validity of statistical data. Krikelas, the 
mDst articulate spokesman for this point of view writes. 

No obvious measurements can be made to determine how effective 
the library is in providing the services necessary to meet its ob- 
jectives. It is important, therefore, to avoid complacency because 
standard meanings for current measures may soon be agreed upon, 
when in reality such measures have no real meaning in relation to 
our objectives .... It is .. the responsibility of each in- 
dividual to recognize that qualitative inference cannot be made 
from such data. Descriptive statistics offer no easy road — in 
fact, no road at all--to evaluation of the quality of library ser- 
vice offered by a given library .... The very nature of library 
administration demands that the librarian engage in resourceful and 
meaningful investigation of the services needed by individuals in 
( his community and the efficiency and effectiveness of his library 

in meeting these needs. 20 

The problem is to enunciate this "resourceful and meaningful investiga- 
tion." As Wright points out, qualitative measures as currently used and 
discussed are primarily value judgments unsupported by empirical data. 21 

To bridge the quality-quantity gap, librarians have come up with 
various schemes. One favorite is the checklist method of analyzing 
collections. If the library owns most of the titles on the list, it is 
presumed to have a quality collection. 

Over 30 years ago, Waples suggested that all books published the 
previous year be divided into three levels of excellence based on group 
judgment. The titles acquired by the library would be compared with those 
at each level to determine the quality of the collection, and then the 
. circulation of titles at each level would be checked. "The results will 
show the level of excellence which the circulation attains in each class 
of publication and each distinguishable group of readers. "22 

He goes on to say that the values of reading change according to what 
is read and who is reading it. This idea is not'unique to Waples; a number 
of others have made similar statements in the literature. Those who propose 
, "quality" evaluations of this sort hold that reading £er se is good— but 
that certain kinds of reading are better for society as a whole as dis- 
( tinguished from an individual preference. 
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Also, as a result of limited resources in nearly evnery locality 
forcing choices in the c'ientele to be served, the profession has gradually 
developed a concept that the reading of some persons has more value socially 
(politically) than the reading of ether persons. Therefore, circulation 
to the elite group or circulation cf quality titles should be weighted 
higher than other items. These san^e kinds of arguments are used in dis- 
cussions of reference services, i.e. , that some questions are more impor- 
tant than others because of the status of the information seeker omhe 
use to which the information will be put. 23 

From a practical point of view, it would be virtually impossible to 
devise and implement an accurate methodology to vveight circulation or re- 
ference service according to the status of the patron, the quality of the 
items read, or the utility of the information provided. 

The third argument, that currently reported statistics are not mean- 
ingful measures is relatively recent and may or may not be valid. Wright 
holds that most of our measures are of resources not services. 24 

Analyses of the kinds of data requested on library reporting forms and 
subsequently published by individual libraries, state library agencies, and 
the federal government support Wright's contention. Almost all published 
statistical data are measures of input, i.e., capacity to provide service. 
Included here would be total holdings, number of staff, volumes added, 
operating budgets and capital expenditures. Population served is simply an 
uncontrolled variable. "Circulation" is the single measure of output 
or service regularly given. 

Using circulation as the sole statistical measure of library output 
implies that it is either the most important service offered by libraries 
or the only service that can be measured. This assumption may not be valid. 
Also, traditionally, circulation figures have been subject to easy mani- 
pulation by shortening the loan period or supplying titles which stimulate 
mass use. Circulation figures tell only how many items were checked out 
of the building; they give no indication of the actual use of these items. 

Most of these arguments stem from the fact that librarians are un- 
familiar with the nature and purpose of statistical analysis. It is con- 
ceded that 100 percent accuracy in reporting figures is almost impossible 
to obtain, even if everyone agrees to count in a uniform manner. This is 
really not an insurmountable problem. As Beasley points out, 

Librarians' work by its very nature deals in large numbers. From 
such large numbers, trends or general emphasis can be determined 
accurately since minor deviations are either absorbed without causing 
a significant change in final results or they cancel each other out.-^ 

Hughes sums up the accuracy controversy by saying. 

Most librarians whE* have been concerned with library statistics seem 
to overemphasize tte detailed accuracy that they feel is essential if 
valid use is to be made of the comparisons. If in other sciences the 
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world were wait ng for absolute accuracy in sampling, as library 
science waits, -lan would not be circling the moon ... The handlinq 
of normal error is normal for competent statistical study As a " 
matterof fact, statistics were developed because it is impossible to 
be entirely uniform and consistent, to count and record every item 
neatly and precisely. Statistics amount to the best guess we can' 
get, on the best information we can get. And that is considerabley 
better than an opinion based on nothing . . . While correctinq- tiie 
flaws in our gathering of library statistics, it is advisable to 
remember. that a three percent margin of error is allowed in matter 
ot life and death and organizations risk capital regularly and 
successfully on a five percent chance of error. 26 

Inferential Statistics 

Semantic confusion exists over the term statistics. The profession 
collects numerical data on collections, staff, budget, etc. The. data 
usually are merely listed in a report. These numbers are descriptive 
statistics. But the method of drawing conclusions from these data is 
inferential statistics. Here is the nub of the statistics controversy. 
Librarians have not yet developed any accepted methods to ascertain the 
validity of a statistical reporting system because they have generally not 
gone beyond simply describing the quantities. yene.diiy not 

infpJ^H.f ^nf? °- literature uncovered only two studies which applied 
Rnlh D traditionally reported library statistics. Charles 

buda- - H'n"°S f a.^iu tiple correlation study using population and 

pndPnJ ^'"V!!'''"'^'' circlilation as indep- 

endent^ variables . The purpose was to find the "best variable to classify 
libraries by size. Budget was deemed the best indicator of library size. 

The correlation test which they used has two inherent problems First 
npS Ic^'''.-'?? strength of linear relationships. This 

method essential y measures only the difference in the extreme ends of the 
trn^u TT^;^ • l°^''.3nd high points. Including in the sample both very 
small libraries having budgets under $100,000 and very large libraries 
nfihP S^t','^'"^'"^ $3,500,000 to $9,000,000 lengthened the slope 

All ^fi.! : 4"^^^ regression also assumes that the budoet correlates 
th^i 2 E"''^"" °^ ^'^^ variables. The other serious problem 

for .?.ff ^^on nnnT' '^'^^ Population, ±15 

Tor staff, ±290,000 for volumes and ±426,000 for circulation. 

n.f./'jr^-i"'^ -"3^ ^^^^ ^^^^ different if they had initially seqre- 

gatedthe libraries by some size factor and applied step^wise linear re 
gression to. change the functional form of the dependent variable. 

fn fho^c?!:-"!-"" appUed multiple regression and factor analysis 

^LJ academic medical libraries. They concluded 

that there were few significant relationships between demand variables (stu- 

No? cou" d^'-^ -nl ?-k' '''' collectfin 5 et) 

mif w-rlh f ^'w'^ variables be closely correlated with service out- 
put Variables (ci rculati.on , ILL and reference statistics) .28 
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An analysis of public library statistical data prepared for this 
study essentially revealed similar findinrs: that population has little 
effect on resources and that resources and output are not necessarily 
related. ^ 



Research on Library Effectiveness 

The literature is replete with articles entitled: "Eval uation'of 
Library Services", "Measurements in Library Service", "Quality Values of 
Library Service", "Indices of Effectiveness of Public Library Services 
in Depth.' Unfortunately, none of the authors fulfilled the promise 
offered by their titles. They offered no formula but simply exhortations 
to strive for effectiveness or to use techniques which have been proven un- 
satisfactory from past experience. All of these titles refer to public 
libraries. Publications aimed at special and university librarians 
have their share of these types of articles too. However, some meaning- 
•ul work has been done in special and university libraries, much of 1t 
by persons outside 1 ibrarianship. 

The most important studies have utilized two techniques borrowed 
from industry and military organizations. These are 1) systems analysis 
and 2) operations research which rely heavily on statistical methods 
Systems analysis is a functional process v/hich segregates and delineates 
the individual functions of an organization. It shows the interfaces 
between functions and their relationship to overall objectives. Operations 
research uses a systems orientation based on the idea that the activity 
of any part of an organization has some effect c,i the activity of every 
other part Therefore, it is necessary to identify all significant inter- 
actions and to evaluate their combined impact on organizational performance 
as a whole. This involves the development of mathematical models and 
simulations of various sub-systems. 

Because some areas of library operations lend themselves more readily 
to quantification and hence are amenable to the construction of models 
these areas have been prime targets for investigation by researchers. 

4. u \ indicates the major areas in which researchers have attempted 

to build models in order to optimize performance and thus influence 
effectiveness. Table II shows the type of library funding agency and back- 
ground or the researcher. The major strengths and weaknesses of the most 
important studies are outlined below. 

_ Philip M. Morse wrote a book in 1968 based on projects done by students 
in his operations research classes at M.I. T. Morse's criteria of ner- 
formance was "unsatisfied document demand." As a result, the book focused 
primarily on book use and its implications for satisfying both current and 
future demands for material. He showed the interaction of demand to cir- 
culation, weeding, and dupTi cation .28a 

4. u'^'^M T°^^^^ presented can be used as a data base for policy decisions 
at the M.I.T. science li.brary. The formulas could probably.be adapted 
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to other libraries once preliminary data had been collected. A major aiii' - 
vantage of Morse's method is that the models can be updated without oatfft^r- 
ing a lot of additional data because they are based on probabilistic' 
occurences. In fact, it would be possible to re-study only selected areas 
of a library's operation once the initial data had been collected. How- 
ever, it should be noted that Morse assumes that the demand rate will re- 
main constant. Thus, the expectation of future- demand in the models is 
predicated on past demand. .. •' ... . 
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Pings, Orr, Pizer and Olson attempted to measure the effectiveness j 
academic medical libraries. User satisfaction was chosen as the ultimas- 
test of library effectiveness. The following criteria were selected to 
reflect user needs: 

1 . Obtaining documents 

2. Locating citations 

3. Receiving answers for specific needs 

4. Having access to work space and facilities 

5. Obtaining instruction and consultation 

Librarians who simulated the user population were to obtain documents 
from a prepared list. The physical availability of the documents was 
scored on a capability index. The limitation inherent in the index is 
that the "score" repr esents a library's abil ity to deliver documents if 
its collection was not being used , i.e . , whether a title is owned re- 
gardless of its actual accessibility to a patron. 

Information services were tested by giving reference librarians 50 
incomplete or incorrect citations to verify within a four hour period 
Random alarm mechanisms (RAM) were used to sample staff activities. 

The 1968 progress report describes checklists of library policies 
which may be important to users. These services are weighted to give a 
quantitative score. The checklist was used only with pre-test groups 29 
No further reports of this study have been found in the literature. 

However, Olson modified the checklist in his study of service policies 
in public, academic, special and school libraries in Indiana. This is the • 
only study located which included a cross-section of types of libraries and 
also a large sample-over 1,000 libraries, half of which were school li- 
braries , 

There are serious questions about using this checklist to measure ser- 
vices. The librarians weight the importance of each service. Their op- 
inions may or may not coincide with that of their users. This method 
relies solely on the veracity of the person checking the form. A service 
that is infrequently provided may be checked for maximum service, yet "if 
the service were more frequently requested it might be curtailed or di^-eted 30 
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Morris Hamburg of the Wharton School of Finance developed "an allo- 
cation model" to determine funding for the 49 outlets v.-hich comprise the 
Free^^Library of Philadelphia. The model is based on "document exposure 
time"--i .e. . the time spend reading both inside and outside the library. 
This model also includes such variables as population served, registered 
borrowers, circulation, in-library use, telephone queries, attendance, 
physical facilities, document resources, and educational level of the . 
population. Reference service is considered only in terms of time and 
money. J I 

All of these factors are related to costs. Hamburg concluded that 
each exposure hour has a value of lit or per person. The methodolooy 
is based on the questionable assumption that people who return books can 
accurately remember how long they spent reading each title. Hamburg 
claims this was 2.25 hours per title. If one analyzes this claim, the 
naievete of this method is readily apparent. If a person read at the rate 
of 500 words per minute— a good rate, he could read 3,000 words per hour. 
If the average book contains 350 words per page that would mean the average 
number of pages read was 19.2 ■ 

Hamburg made no attempt to find out whether circulated books were read 
completely, partially or not at all. 

The Univers.ity of Lancaster studies have been primarily concerned 
with modelling the university library's ability to satisfy users 's needs 
for materials in terms of circulation policies, availability of documents, 
duplication and time required to process in-cominq materials, weeding, 
journal purchasing and inter-library loan. A "frustration survey" similar 
to the Capability Index was conducted among actual users. 32 

The Lancaster study is particularly interesting for two reasons. 
The researchers are librarians who have an advanced knowledge of statistics. 
Secondly, the models were actually implemented and library operations 
were changed as a result. This project demonstrates that research can 
have a practical and beneficial payoff. 

Two university library cost benefit studies are Raffel's and Shishko's 
at M.I.T. and Durham's and Newcastle's in England conducted by the Durham 
Computer Unit. The M.I.T. researchers calculated costs for: 

1. open and closed access book storage 

2. book versus microform storage or xe"ox 

3. various types of reserve systems incl uding' cheap xerox, free xerox 
or microforms 

• 4. seating 

5. temporary cataloging 



6. rapid inter! ibrary loan 
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7. weeding 

After present operational costs had been determined along with costs 
of changing the existing system, a questionnaire was sent to a random 
sample of faculty, graduate students and undergraduates. They were asked 
to rank which services they would prefer to keep or change within the 
limits of a $200,000 budget increase, a $100,000 increase and no increase. 

Although answers are not really applicable to public library operation, 
the findings illustrate the variegated nature of user preferences. The 
undergrads wanted centralized reserve, the graduate students wanted more 
books to check out while the faculty prefered departmental libraries. 
Obviously satisfying any one group would be at the expense of the other 
two . • 

The title of the Furham study, "Projects for Evaluating the Benefits 
from University Libraries", promised more than it delivered. 34 Benefits 
are cast in terms of unit costs and the relationship between the cost of 
one service vs_. the cost of another. For example, the cost of adding one 
book is equivalent to obtaining 4.5 items on ILL or circulating 90 books 
on long term loan. 

The premise is that once the library administrator has detailed cost 
and volume of activity data, he can better decide which services he wants 
to expand and which he wants to curtail to provide an optimum balance. He 
may or may not take needs or even demands into consideration. By this 
method the library administrator evaluates the benefits offered by the 
library on the supposition that he is the best judge of "good" service for 
his institution. 

Weeding, storage and duplication have been analyzed by Trueswell , 
Fussier and Simon, Jain, and Leffler. They essentially view the library 
as an inventory supply problem and try to assess which materials will be 
called for most frequently and which will languish on the shelves. AVI 
have found that the .probabi 1 i ty of a book's being used declines with the 
age. This finding, in turn, relates to Trueswell's 80/20 rule which says 
that 20 percent of a university library's collection accounts for 80 per- 
cent of its circulation. The 80/20 rule may not apply to public libraries 
because of the large percentage of fiction in their collections. 

All of these studies relate to effectiveness in the broad sense in 
that they attempt to analyze some aspect of library service and devise 
ways to improve that service. The similarity among all of them is that 
they focus on availability and use of documents and budget allocation. 

Availability of documents v/as measured by the number of unfilled re- 
quests and the resulting implications for providing duplicate copies or 
changing circulation policies for a more rapid turn over in book stock,. 
Document use was studied not only in the context of availability but also 
in tem^s of browsing and selection oi^ materials for v/eeding and storage. 

The research on budget allocatioin reflects current management interest 
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in accountability and cost benefit analysis. These studies polled users 
and/or library staff to select alternatives to present services within 
the confines of a limited air.ount of money. 

An analysis of the empirical studies completed to date prompts the 
following conclusions: 

1. Most studies were done on individual 1 ibraries--primarily aca- 
demic institutions which may or may not be typical of this group. 

2. The research has not been cumulative. Some aspects of library 
operations like weeding, storage, duplication and unsatisfied 
demand have been done over and over again by different researchers. 
Yet other critical areas have virtually been ignored--reference 
service, the library's impact on its conminity, optimum utiliz- 
ation of staff. 

3. The principal researchers involved in most of the mathematically- 
oriented studies have no library training or experience. As a 
result some of the concepts presented and the approaches tried 
show a naivete of the complex nature of the activities of the 
library. 

4. These studies, also, have limited use at this time in decision 
making because they involve a level of mathematical application 
which is far greater than the accuracy or availability of data. 

5. Host of these reports describe the library as though the library 
staff did not exist. One wonders just what level of involvement 

the staff had while the studies were going on. If staff involvement 
was minimal, it might be hypothesized that the studies had little 
impact on subsequent operations or services. Only the-Uni versi ty 
of Lancaster has written on the actual imp-lementation of the 
models prepared for that library. 

6. Also, no follow-up reports have appeared in the literature showing 
that these models have been adopted and/or adapted by other li- 
braries. 



CONCLUSION 

The results of the literature search are clear. There exists few an- 
tecedent approaches which the public library can utilize fruitfully in de- 
veloping innovative approaches to measuring the performance of the services 
it offers its public. For the most part, most earlier efforts, library and 
non-library, must be characterized as incomplete or "half-way" measures. 
Further, few of the approaches cited lend themselves to implementation and 
interpretation by the librarian because of their reliance on highly sophis- 
ticated and cprnplex methodologies. 

The few exceptions .to the above oberservation are also limited in their 
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usefulness to the public'librarian who wants to utilize a fsirly broad- 
based program for evaluating and therefore measuring library services. 
The approaches advocated so far have been seen as too esoteric or too' 
complicated or too remote from reality or too simplistic or too narrov/ 
or too broad or too ... . 

Too often, also, the researchers in this area have worked in a vacuum, 
not sufficiently sensitized to the world of library operations to make- the 
crucial connection between methodology and meaning of importance. Or, too 
often, the marriage between researcher and practitioner has been so close 
that objectivity of what and why this or that is being measured never sur- 
faced. In general, there has been an absence of adequate transmittal of 
knowledge from the techniques used to the individuals responsible for their 
final interpretation and operation. 

Clearly, if the researcher's interest goes beyond additional insight 
into the problem of "how to measure" and includes the concern of acceptance 
and utilization of the given scheme developed, then an understanding of 
the politics of measurement is essential. Foremost here is the general 
skepticism about any statistical approach which purports to measure quality. 
Secondly, there are the pressures from outside agents to conform to de- 
vices which they, the agents, have devised as reliable indicators of 
performance. The ambivalence towards statistics, despite or because of 
its'historical antecedents, has helped to create a vacuum today which ex- 
plains the dilemma in which most library decision makers find themselves. 
For the most part, the profession is clearly in no stronger power position 
as a result. 

In short, those researchers who have dealt with the problem of statis- 
tical measuremerit have not been successful in communicating their schemes 
to those under operational fire. They have not proved ^:he case for either 
the relevance or operational meaning of the approaches devised, but have 
tended to act as if the world of statistical measurement is crystal clear 
to everyone and its meaning instantly . obvious . They have assumed erroneously 
that the adoption automatically comes from explication. 

Perhaps C. West Churchman is correct when he concludes in his book 
that ''ultimate meaning of the systems approach, (measurement approach?)... 
lies in the creation of a theory of deciption and in a fuller understanding 
of the vvays in which the human being can be deceived about his world and 
in an interaction between these different viewpoints." 

The thrust of Churchman's argument 'is that the planner or researcher 
has not really faced up to the multiple factors, rational or irrational, 
which significantly undermine their schemes. "... among the anti -planners 
there is the completely non-intellectual approach, the approach that does 
not believe that thinking in any of its senses is important in the develop- 
ment of human life. It is the approach that finds the essence of value 
in the song, the painting, the vision, the myth, the feminine, and ultimately 
the unspoken. What is not said at all is the most important thina of all. 
Since the management scientist, the planner, and the behavioral scientist 
spend all their time speaking, then it must be the case that what they spend 
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their time on is the least important part of human lift!i.*' 

We suppose one would encounter a more receptive group if a total con- 
demnation of statistics and the prospect of relating quality with quantity 
prefaced all ensuing comments. We could then concl ude . that despite its 
inherent weakness, our statistical approach v;ould clearly be the lesser 
evil and should be given a try. At the very least, we guess, we would 
have made our contribution towards the art of deception. The unfortunate 
case is the prevalent confusion between the abuses of the technique with 
the logical foundation of its existence. Rationally we know that quantity 
infers something about quality —a reading room with one seat or a library 
with one book suggests certain limitations on the quality of those opera- 
tions! --Emotionally, of course, we resist any such connection. And too 
often it is just that emotion which determines our behavior in the final 
analysis. .1 
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Kenneth E. Beasley and Ernest R. DeProspo 



INTRODUCTION 



The United States Office of Education has been publishing 
a series of selected statistical data for public libraries 
since 1944.1 These statistics are used ostensibly for sever- 
al purposes: 

1. To provide an internal statistical record for 
individual libraries and as an easy way to 
compare similar libraries 

2. To assist the federal government in decision making 

3. To assist the public in evaluating the overall 
development of public libraries 

In practice, the comparison of libraries has been used 
extensively to show deficiencies in library service indivi- 
dually or collectively and consequently to justify a need 
for greater financial -apport. Implicit in this use, also, 
is that numerical quantities bear some relationship to 
actual performance or af f ectiveness . Larger quantities 
in almost all areas of service have commonly been presumed 
to mean better service. Although the validity of this 
relationship has been discussed widely in the literature 
for many years, the criticisms have generally focused on 
the need for so-called qualitative measures of performance 
rather than v?ith substantive critiques and development of 
the quantitative measures themselves. 

In 1971, the U. S. Office of Education authorized a • 
study to develop measures of or criteria for public library 
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effectiveness. This project v.-as sponsored by the American 
Library Association and conducted by the Bureau of Library 
and Information Science Research at Rutgers University. In 
this study, current statistical reporting systems were 
reviewed to determine their utility and to make preliminary 
judgments about the need for new measures. Since nearly all 
state reporting systems are modeled after the U.S.O.E. 
statistical reports, the latter were analyzed in detail. 

U.S.O.E. statistical reports for public libraries are 
totally descriptive. Numerical quantities are listed in 
each category with no attempt to analyze what these quantities 
mean or how they may be related to each other. In essence, 
the U.S.O.E. statistical report is a census of public lib- 
raries. Although modifications have been made in the data 
base in each publication, certain traditional categories 
have remained constant: (a) population served, (b) size 
of collection, (c) total operating expenditure, (d) number 
of professional and clerical employees, (e) number of vol- 
umes added the previous year, (f) salary expenditures, 
(g) library material expenditures, and (h) circulation . ^ 
Of these items, all except circulation and population are 
measures of input . Population is neither an input or an 
output. It is an uncontrolled variable. Circulation as 
traditionally defined reflects the only attempt to determine ■ 
output. For all of these items, accurate counting is 
essential if comparisons are to be useful or if totals are 
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to be indicative of overall regional, state or national 
development. Hov/ever, all parties concerned have not been 
able to agree on what constitutes the data categories or 
how they should be counted. in the absence of general 
agreement comprimises have been made v;hich reflect the- lowest 
common denominator. 

While U.S.O.E. has always recognized this fact, it 
has of practical necessity been required to publish the 
data essentially as submitted by the individual libraries 
and has admitted that much greater uniforraity would be 
desirable. This same problem v;as encountered in this analy- 
sis and could only be handled by assuming accuracy in report- 
ing; any deficiencies found in using the statistics as 
' measures of effectiveness would therefore be accentuated 

if the data itself were defective.^' 

.11 

The library profession has traditionally operated on 
a basic premise that "more equals better." One can find 
hundreds of examples in the literature stressing the need 
for increased funds to provide "better" service. Impli- 
citly and explicitly, the assumptions are that money buys 
larger collections, more professional < and clerical staff, 
more service units; and from this there is increased cir- 
culation and generally a better quality service. To test 

ERIC 
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these assumptions, the U.S.O.E. statistics v;ere analyzed 
by a stepvjise multiple regression and factor analysis. 
The objective was to determine predictive variables--what 
inputs produced what outputs.^ 

Since the Rockwoods ' study in 1967 had determined that 
budget was the best indicator of library size/ a sample of 
180 public libraries stratified by budget and geography was 
drawn from Statistics of Public Libraries Serving Areas with 
at Least 25, 000 Inhabitants . 1968 . 

Number of Libraries 

Class of Library Amount of Budget in Sample 

I. Small Libraries $100,000-$ 249,999 71 

II. Medium Libraries $250,000-$ 749,999 61 

III. Large Libraries $750 , 000-$ 3 , 4 9 9 , 999 48 

180 

Libraries with budgets less than $100,000 and more than 
$3,500,000 were omitted from this sample (and other phases 
of the total study) because the very large libraries are 
highly individualized and other studies have shov/n that the 
very small libraries appear to be almost a distinct kind of 
institution . 

For each of the .mple libraries for the year 1968, the 
following data were collected from- Statistics of Public 



Libraries 
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Total Operating Expenditures 
Population Served 

F.T.E, Library Staff Positions, excluding maintenance 
staff 
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Staff holding M.L.S. degree 

Total Salary Expenditures, excluding itaintenance 
salaries 

Expenditures for library materials 

Number of branches 

Total book and serial holdings 

Total book and serial volumes added during fiscal 
year 

Total circulation (transactions of all materials 
lent for use outside the library during fiscal 
year) 

Factor analysis was applied separately to each of these 
classes of libraries with the following results:^ 

No consistent pattern could be discerned. A variable 
might fall under factor 1 for the large class (I), under 
factor 3 for the medium (II) , and under factor 2 for the 
small class (III) . 

The next step, accordingly, v;as to use a stepwise 
multiple regression program (v/hich constructs a prediction 
equation one independent variable at a time by selecting the 
independent variable which is the best predictor of the 
dependent variable) . This program permits one to then add 
the other variables step-by-step in order of importance until 
no other variable will contribute significantly to the 
prediction equation. ■'-^ • 

The program was run separately for each of the three 
classes of libraries and for the entire sample. Each of the 
10 variables v;as dependent in each set. The results were: 
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1. The variables appear to be redundant in that they 
reflect different facets of the same measure-- 
total operating expenditure. As a result the 
correlations among these variables tend to vary 
together in a consistent pattern. . 

2. Standard errors for all calculations were uniformly 
high indicating a great deal of "uncertainty" in 
the predictions. 

The final step in this analysis was to determine whether 
significant differences exist between the different library 
size categories. To determine this, twenty-four ratios v;ere 
computed for each individual library by dividing one variable 
by another. The ratio categories are given in the appen- 
dix. The ratios were then ranked in each category for the' 
entire sample and for the size-of -library sub-samples. An 
analysis of the ranked scores indicated that the data v;ere 
not normally distributed. Also the ranges and standard 
deviations v;ere large for many categories. 

Therefore, on the advice of a statistician, it was 
decided to use the median test. This is an application of 
chi-square to ordinal data to determine whether a signifi- 
cant difference exists in the scores of tv;o or more samples. 
The common median for the entire sample is determined. The 
number of libraries within each class scoring above and 
below the common median is counted. Then the chi-square 
statistic is calculated. The chi-square statistic and the 



significance level for each ratio category are shown in the 
appendix . 

Of the 28 ratios compared across the three classes, 
only 12 showed significant differences at the 0.05 level 
(See Table I). Nine of these 12 significant relationships 
dealt with some aspect of finance. Although large libraries 
spend almost $1.00 per capita more than the small libraries 
and 57<? more than the medium libraries, there are no 
significant per capita differences in: 

1. volumes added 

2. service units 

3. amount spent for library materials 

4. circulation 

5. holdings 

6. library staff 

7. M.L.S. staff 

The logical question is where then does the money go? 
The answer is salaries. Large libraries on the average 
spend 52% of their budget for salaries, medium institutions 
61% and small libraries 56%. The average annual salary in 
large libraries is $410 higher than in the medium libraries 
and $77 0 higher than in the small sized. Both the fraction 
of the budget spent for salaries and the amount of the median 
salary were significant at the 0.05 level. 

One might logically hypothesize that the difference in 
salaries could be attributed to the presence of a higher 
proportion of professionals (M.L.S. staff) in the larger 
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libraries.l2 Hov;ever, the differenco in the number per 
professionals related to total staff vms not significant. 
How then did the number of M.L.S. staff relate to other 
variables? In general, small and medium sized libraries 
had larger collections, added more volumes and circulated 
more materials per M.L.S. staff m^ember. Hov;ever, these 
statistics do not reflect specialized activities performed 
by professional staff in many libraries. In terms of total 
staff, only holdings per staff member proved significant- 
small libraries owne: about 3, 880 more vol-omces per employee 
■ than the large libraries while medium sized institutions 
owned 6,480 more than the large. 

The relationship of holdings to circulation is signi- 
ficant in that large libraries had the lowest ratio of cir- 
culations per volume ovmed. Interestingly, there were no 
significant differences between circulation and the number 
of volumes added or circulation and the amount of money 
spent for library materials. 

Thus, using only- the implied assumptions noted above, 
a statistical comparison of libraries of different sizes 
based on the data categories used in the U.S.O.E. report 
suggests that small libraries give a greater return per 
dollar spent, and that the economy of scale normally ex- 
pected in larger institutions is not evident. Although 
this finding appears to contradict current economic theory, 
it may be a result of limitations in the data categories in 
the U.S.O.E. report rather than in the fiscal management of 

O 
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the libraries studied. Hov;ever, the preliminary results 
of the v:harton School Report (Morris Hamburg) suggests a 
similar conclusion by using a corr.pletely different statis- 
tical concept. 

Yet, such a conclusion v;ould not be accepted by the 
most severe critics of library service, and it is here that 
the librarians have justifiably fallen back on the demand 
for qualitative criteria which reflect the intangibles of 
a reference service or the differences in legal 'or board" 
articulated functions. The lack of significant differences 
among so many items could mean, among other possibilities, 
that libraries (a) have many more discrete developmental 
items than suspected in the past, or (b) libraries of dif- 
• ferent sizes or stated functions are much more similar than 
everyone believed, or (c) there are only a few true deter- 
minants of the form and quantity of library service. None 
of these conclusions is completely acceptable because actual 
inspection of libraries shows an almost unbelievable range 
of alternative developmental patterns. 

Since this analysis is only a preliminary one using 
certain preset parameters, one can only say that 
statistics as now collected appear to have a very limited 
value (a) in making valid comparisons, (b) as a basis for 
setting standards of development or performance, or (c) 
to establish historical trend lines. Why they have limited 
usefulness can only be hypothesiEed . No one has ever made 

( 

o 
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a conscientious effort to develop a theoretical or errrir- 
ically based justification for this set of data. This set 
of statistics hov/ever, does provide clues for formulating 
a more sophist.icatGd system of statistics; and to this 
extent, plus the o^bsence of other -alternatives of measure- 
m^ent, they are an acceptable first qencration tool- This 
pro>rct Vj^s the objec ive ji ueveloping a more sophisti- 
cated second generation model. 



*A more detailed theoretical analysis of the U.S.O.E. 
statistics v;ill be written as a later part of this study. 



Although the Bur^.i c he Census began collecting 
libr^rv statistics in 1850 zy-:i: ^irs^ major publi cation v/as issued 
by the Office of Education in IS'76. 

^See Salversca, 'relevance of Statistics to Library 

iEvaluation. " College and Research Libraries 30:352-351 ^ 1969 f t)r ' 
a discussion on this point. 

"^An initial plan to c/'^o chart trend data had to be aban- 
doned because U.S.O.E. chanr;:'>H . data base each time a new report 
v/as ir^^i/'^d. 



'"If the data fs entr'^'./^' ./rong, then there is ipso facto 
no need to collect it. 



All calculations v/ere done by computer using SPSS . . 
g 

Rock:>vi:}iDd , Ruth afi:: Charles Rockwood, Q uantitative Guides 
to fab>1ijc Library Qperationj. Illf^qois, University. Gradual,.: School 
LibjMry Science. Occasional Paper. 1967. 

'^Complete citation: National Center for Educational 
Statistics, Statistics of Public Libraries Ser\niici Areas wi th at_ 
Le_a_st 25,000 Inhabitants . 1968. Washington, GPOfi'lay 1970. 

o 

If possible, at least three libraries were selected frtra 
each state, except Hawai'h for each budget class. 

0. 

"Siie, Noriw. Dalfe H". Brent, and C. Hadlai Hull. SPSS : 
Statistical Package for tht S ocial Sciences . New York , McGraw-Hi 1 1 , 
1970. pp. 209-210. ' ' 

""^For a fuller eccpilanation see: Nie, et a1 , Ibid . , pp. 180-181. 

"'"'since data on IE variables were available, there were 55 
possible ratio categoriES.. However, 21 of these ratios were meaning- 
less . 

1 ? 

"-For further disuussion of the median test see John T. Roscoe, 
Fundamental Research Statistics f the_ Behavioral Sciences . New York : 
Holt, Reinhart & Winston, 1969. .pp. 201-202. 

^'^It was previously stated that M.L.S. staff per capita was • 
not significantly different among the three groups. 
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Mean (circulations) 
Standard deviation 
Range 
Median 

Significance level 



. CIP.CUL'iTION/VOLUiMES ADDED 
(For every volur.- added n books circulate) 
Total=l80 Larae=4 8 Mediuir.= 61 



Smll=71 



39,977 
17,853 

121,455 

_ 37,054 

Chi square=2 



37,060 
17,720 
121, 460 
35, 750 



49,9 70 
16,700 
80,000 
37,160 



2DF Not significant at 0.05. 



41,100 
18,900 
80,340 
39 ,000 



HOLDINGS/VOLUMES ADDED 
(Ratio of volumes owned to volumes added) 
Total=180 Large=48 Medium=61 



Small=71 



Mean (volumes) 13.381 
Standard deviation 6.42 
Range 54.00 
Median _ 12.42 

Significance level chi square=3.1 



12.97 
5.50 
34.00 
12.37 



14 .28 
7.41 
50.50 
13.40 



12.88 
6.06 
32.26 
11.38 



2DF Not significant at 0.05. 



POPULATION/VOLUMES ADDED 
(One volume added for every n persons) 

Total=18 0 Large=4 8 Medium=61 



Mean (Persons) 
Standard deviation 
Range 
Median 

Significance level 



Small=71 



8,057 
4,680 
37,333 
7,023 
chi square=2,0 



7,460 8,090 8,430 

3,930 5,660 4,220 

26,550 35,990 22,590 

7,180 7,320 6,840 

2DF Not significant at 0,05, 



POPULATION/SERVICE UNITS 
(Average population per service unit) 

Total=180 Large=4 8 Medium=61 



Small=71 



Mean (persons) 
Standard deviation 
Range 
Median 

Significance level 



34,070 32,100 35,010 

21,022 15,190 24,900 

113,545 64,730 110,000 

27,983 28,610 26,190 

chi square=2.5 2DF Not significant at 



34,600 
20,970 
96 ,550 
28,750 
.05. 



CIRCULATION/PRINT IWERIALS EXPENDITURES 
(For every $ spent on print materials n books circulate) 

Total-180 Large=4 8 Medium=61 Small=71 



Mean (circulations) 11.62 

Standard deviation 35.76 

Range 483.81 

Median 8.39 

Significance level chi square=2.3 



8.62 
2.79 
12.04 
8.22 



8.96 
3.82 
18,45 
8.34 



erJc 



2DF Not significant. 



15.94 
56.76 
483.81 
8.88 



PRINT MATEPJ7vLS e>:pen'ditupj:s/volu:-:i:s added 





(Average cost 
Total=180 


per voluiT-.e) 
Larae=4 8 


V o o "i viT» = 6 1 


Sinall=71 


Mean (cost) 


$4.63 


$4.38 


$4.90 


$4.57 


Standard deviation 


1.76 


1. 88 


1.94 


1.48 


Range 


15.27 


15.27 


13.12 


9.63 


Median 


$4.43 


$4, 33 


S4.77 


$4.36 


Significance level 


chi square=l . 1 


2DF Not si 


cr.if icant . 


A. • • 



SALARIES/TOT/vL EXPENDITURES 
(Fraction of the budget spent on salaries) 

Total=180 Large=4 8 Hedium=61 Sniall=71 



Mean (percent) 


59% 


62% 


60% 


57% 


Standard deviation 


0 


.08 


0. 


07 


0. 


08 


0. 


08 


Range 


0 


. 432 


0. 


318 


0. 


394 


0. 


393 


Median 


0 


.591 


0. 


617 


0. 


511 


0. 


564 


Significance level 


chi 


square=9 . 6 


2DF Signif 


ican 


t at 0.01. 







PRINT r-lATERIAL EXPENDITURES/TOTAL EXPENDITUPJES 
(Fraction of the budget spent on print materials) 

Total=180 Large=4 8 Medium=61 Sinall=71 



Mean (percent) 


18 


. 6% 


16 


.1% 


18 


.7% 


20 


.3% 


Standard deviation 


0. 


058 


0. 


0,46 


. 0. 


061 


0. 


056 


Range 


0. 


420 


0. 


198 


0. 


353 


0. 


384 


Median 


0. 


178 


0, 


154 


0. 


172 


0. 


197 


Significance level 


chi square=13.8 


2DF Signi 


f ica 


nt at 0. 


01. 





PRINT MATERIALS EXPENDITUPIJS/S/iLARIES 
(Ratio of print material expenditures to salaries) 

Total=180 Large=4 8 Medium=61 Small=71 



Mean (ratio) 


0 


. 329 


0 


.267 


0. 


327 


0. 


.373 


Standard deviation 


0 


. 136 


0 


.097 


0. 


144 


0. 


.135 


Range 


0 


.966 


0 


.406 


0. 


875 


0. 


.744 


Median 


0 


.311 


0 


.245 


0. 


287 


0. 


.344 


Significance level 


chi 


square=10 . 9 




2DF Signif 


icant at 0 


.01. 





TOTAL EXPENDITURES/POPULATION 
(Expenditures per capita) 

Total=180 Large=4 8 Medium=61 Small=71 



Mean (dollars) 


$3. 


96 


$4. 


33 


$4. 


50 


$3. 


25 


Standard deviation 


2. 


11 


1. 


78 


2. 


76 


1. 


35 


Range 


. . 15, 


15 


7. 


47 


14. 


99 


5. 


86 


Median 


3. 


53 


4. 


15 


3. 


52 


3. 


17 , 


Significance level 


chi square=7 . 2 


2DF 


Significant 


at 0.05. 







SALARIES/LIBRARY STAFF 
(Average salary per staff ir.ember) 





Total.=180 


LargG=4 8 


!-'ediu~=61 


Small=71 


Mean (dollars) 


$4 ,912 


$5,410 


$5,000 


$4 ,500 


Standard deviation 


1. 073 


1.020 


0.960 


1.050 


Range 


• 6.566 


4 , 880 


5 .5 30 


^- • 6.570 


Median 


4.845 


5.340 


4.930 


4 .570 


Significance level 


chi square=9 . 6 


2DF Significant at 0 


.01. 



PRINT MATERIALS EXPENDITUPJ-S/POPULATION 
(Print materials expenditures per capita) 

Tota" -a, 8 0 Large--=4 8 Med iu^ii^ 'I*' Small=71 



Mean (dollars) $0.71 $0.65 $0.81 $0.65 

Standard deviation 0.40 0.23 0.55 0.30 

Range 3.13 1.03 3.00 1.68 

Median 0.62 0.61 0.65 0.63 

Significance level chi square=l . 3 2DF Not significant. 



CIRCULATION/POPULATION 





(Circulation ] 


oer capita 


,) 






Total=180 


Large=48 


Mediura~61 


Small=71 


Mean (circulations) 


5.91 


5.59 


6.35 


5.75 


Standard deviation 


3.10 


2.66 


3.33 


3.18 


Range 


16.50 


12. 04 


15.40 


16.14 


Median 


5.24 


5.03 


5.35 


5.20 


Significance level 


chi square=4 . 0 


2DF Not 


signif i cant . 






HOLDINGS/POPULATION 








(Books per 


capita) 








.Total=lBO 


Large=48 


Medium=61 


Sraall=71 


Mean (books) 


1.94 


1.96 


2. 19 


1.71 


Standard deviation 


1,10 


0.94 


1.64 


0.73 


Range 


10.50 


5.24 


10.09 


4.04 


Me -^ian 


1.76 


1.74 ' 


1. 93, 


1.67 


Significance level 


chi square=3.8 


2DF Not 


significant , 
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POPULATION/LIBPJvRY ST/vFF 
(Population per staff member) 

Total=180 Large=4 8 Mediuiii=61 



Small=71 



Mean (persons) 2,642 
Standard deviation 1,410 
Range 8,69 3 

Median 2,2 83 

Significance level chi square=0.7 



2,390 2,390 3,030 

1,090 1,120 . 1,720 

4,450 6,040 8,440 

2,170 2,220 2,590 

2DF Not significant at 0.05. 



CIRCULATION/LIBPJVRY STAFF 
(Circulations per staff member) 

Total=180 Large=4B Medium=61 



Mean (circulations) 12,822 

Standard deviation 4,492 

Range 24,30 3 

Median 12,29 8 

Significance level chi square=9 . 4 



S.mall=71 



11,580 12,600 13,850 

4,090 3,630 5,190 

23,810 21,010 23,430 

11,140 12,160 13,410 

2DF Significant at 0.01. 



HOLDINGS/LIBRARY STAFF 



(Holdings per Btaff mem.ber) 





Total=180 


Largs=4 8 


Medium=61 


Small=71 


Mean (volumes) 


4 ,226 


4 , 030 


4,300 


4,290 


Standard deviation 


1,359 


1 , 210 


1 ,480 


1 ,350 


Range 


10,940 


5, 820 


10,840 


6,630 


Median • 


4,059 


. 3,820 


4 ,180 


4, 140 


Significance level 


chi sqiaare=4,0 


2DF Not 


significant at 


0.05. 



VOLUMES ADDED/LIBPARY STAFF 
(Volumes added per staff meml^er) 





Total=180 


Large=48 


2-1edium=61 


Small=71 


Mean (volumes) 


349 


330 


340 


400 


Standard deviation 


123 


120 


130 


120 


Range 


755 


630 


670 


580 


Median 


333 . 


320 


330 


370 


Significance level 


chi square=3.1 


2DF Not 


significant at 


0.05. 
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LIBRARY STAFF/i^lI.,S STAFF 
(Number of staff members per professional) 



Total=180 Large=4 8 Hediuir.= 61 Small=71 



Mean (persons) 


8. 10 


7. 


70 


8. 


10 


8. 


38 


Standard deviation 


8. 06 


10. 


26' 


6. 


17 


7. 


91 


Range 


70. 50 


68. 


3 3 


34 . 


GO 


*39. 


00 


Median 


5. 99 


5. 


30 


6. 


38 


6. 


47 


Significance level 


chi square=4..2 


2DF Not 


signif 


leant 


at 0.0 5 





POPULATION/MLS STAFF 
(Population per MLS staff member) 





Total 


=180 


Large = 4 8 


Hediiim=61 


Small=71 


Mean (persons) 


22, 


310 


19, 


440 


19 ,460 


26, 


700 


Standard deviation 


26, 


968 


27, 


950 


19 ,570 


31, 


230 


Range 


148, 


000 


142, 


310 


111,000 


148, 


000 


Median 


13, 


266 


11, 


810 


13 , 720 


16, 


080 


Significance level 


chi square=2.9 


2DF 


Not 


significant 


at 0.05. 





CIRCULATION/MLS STAFF 
(Circulationc par MLS staff member) 

Total=180 Large=4 8 Medium=61 Small=71 



Mean (circulations) 


101 ,189 


89, 


420 


100 


,090 


110 


, 090 


Standard deviation 


39,185 


Tin 

a. , 


580 


/ D 


,8 80 


10 5 


,530 


Range 


668,000 


651, 


550 


360 


,000 


464 


,000 


Median 


69 ,690 


53, 


520 


70 


,240 


76 


,080 


Significance level 


chi square~2 » 9 


2DF 


Not 


signif ica;nt 







HOLDINGS/MLS STAFF 
(Holdings per MLS staff member) 

Total=180 Large=40 Medium=6l Small=71 



Mean (volumes) 


33 


,961 


29, 


590 


33, 


880 


37, 


,000 


Standard deviation 


31 


,407 


31, 


250 


25, 


610 


35, 


,800 


Range • 


187 


,00 0 


179 , 


760 


140 , 


000 


166 , 


,000 


Median 


24 


,0 4 7 


20, 


810 ' 


27, 


290 


24, 


,690 


Significance level 


chi square=7.0 


2DF 


Significant 


at 0 


.05. 
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VOLUMES ADDED/MLS STAFF 
(Volumes added per MLS staff, r.e-her) 

Total=180 Large = 4 8 Hediuin=61 



Sniall=71 



Mean (volumes) 
Standard deviation 
Range 
Median 

Significance level 



2,747 
2,676 
16,600 
2,010 
chi square=3.9 



2,330 2,830 2,960 

2,550 2,620 2,810 

16,600 12,000 -10,000 

1,670 2,190 2,190 

2DF Not sif.mificant at 0.05, 



CIRCULATION/HOLDINGS 
(Circulations per volume ovmed) 

Total=180 Large=4 8 Medium=61 



Small=71 



Mean (circulations) 
Standard deviation 
Range 
Median 

Significance level 



3,189 
1,149 
6,927 
3,062 
chi square=6.9 



3,020 3,080 3,390 

1,130 910 1,320 

5,380 4,510 6,930 

2,930 3,010 3,240 

2DF Significant at 0.05. 
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